Skip to content

Refactor: decompose Parse::parse() into ParseContext + per-state handlers - #71

Merged
mmucklo merged 12 commits into
masterfrom
refactor/parse-decompose
Aug 26, 2026
Merged

Refactor: decompose Parse::parse() into ParseContext + per-state handlers#71
mmucklo merged 12 commits into
masterfrom
refactor/parse-decompose

Conversation

@mmucklo

@mmucklo mmucklo commented Aug 20, 2026

Copy link
Copy Markdown
Owner

Decomposes the ~772-line Parse::parse() state machine into a thin dispatch loop over per-state handler methods, backed by a typed ParseContext object. Pure behavior-preserving refactor — no parsing logic, conditions, ordering, error codes, or output shape changed; the 108-case suite is untouched and output is byte-identical. Fully backward compatible: no public or protected method signature changed vs the previous release.

Before → after

flowchart LR
    subgraph B["Before: one method"]
        P1["parse()<br/>772 lines<br/>one switch, 12 states<br/>~30 loose loop-locals<br/>untyped $emailAddress array"]
    end
    subgraph A["After: dispatcher + handlers + state object"]
        P2["parse()<br/>185 lines<br/>setup + dispatch loop"]
        H["7 state handlers<br/>+ 4 address sub-helpers"]
        C["ParseContext<br/>203 lines<br/>~24 typed fields"]
        P2 --> H
        P2 -. mutates .-> C
        H -. mutates .-> C
    end
    P1 ==>|"decompose"| P2
Loading
Before After
parse() body 772 lines 185 lines (setup + dispatch loop)
State logic inline in one switch 7 handler methods + 4 address sub-helpers
Working state ~30 loose loop-locals + untyped $emailAddress array ParseContext — 203 lines, ~24 typed fields
Methods in Parse.php 19 28
Accumulator access $emailAddress['key'] (array) $ctx->key (typed property)
Reentrancy shared loop-locals fresh context per call, never on the instance

LOC accounting (read past GitHub's diff)

GitHub's line-diff counts relocated and renamed lines as churn, so it reads like a rewrite. It isn't. Scoped to src/:

Metric Value Note
GitHub churn +1,083 / −825 line-diff counts moves & renames on both sides
Net source growth +258 Parse.php +55 (1613 → 1668), new ParseContext.php +203
Master code lines unchanged 622 / 987 (63%) survive byte-for-byte (normalized, non-blank)
Master code lines edited 365 / 987 (37%) almost entirely the two mechanical transforms below ↓
— array → typed property 418 sites $emailAddress['x']$ctx->x
— variable rename 440 sites $emailAddress$ctx
Behavior / output changes 0 108 tests unchanged, output byte-identical

So the honest footprint is +258 net lines and two find-and-replace-shaped transforms — not the ~1.9k-line churn GitHub advertises. The counts are lenses, not addends (the 418/440 are occurrence counts within the 37% of edited lines).

What changed

  • parse(): 772 → 185 lines — setup + a thin switch ($ctx->state) dispatching one handler per state, plus post-loop finalization.
  • New src/ParseContext.php — a per-parse object holding the ~24 accumulator fields + loop control + hoisted input/config as typed properties. Created fresh per parse() call and never stored on the Parse instance, so reentrancy is preserved (a localPartNormalizer callback can re-enter parse()). Its constructor requires the initial state/sub-state, so an un-initialized context is unrepresentable.
  • Extracted handlers (handleStateTrim, handleStateAddress, handleStateQuote, handleStateComment, …) — Parse.php method count 19 → 28; the hot atext/period branches are inlined for performance.

Independent verification (not just the author's report)

  • Correctness: 108 tests pass on default and SEED=1/42/1337/99999; PHPStan level 8, Psalm, and CS all clean. Reentrancy confirmed with a localPartNormalizer that re-enters parse() — returns the correct result, no context corruption.
  • Performance: measured a min-of-5 throughput A/B (refactored vs pre-refactor master), Xdebug off. Refactored is faster — 1.88s vs 3.18s for 125k parses. (The sandbox's default config has Xdebug on + opcache-CLI off, which penalizes method calls; that instrumentation artifact accounts for any interpreted slowdown seen without it.)
  • Max-effort code review: line-by-line equivalence pass across every extracted handler — no correctness regressions. The one intentional change (resetAddress() now zeroing commentNestLevel per address) is inert: an address can never reach STATE_END_ADDRESS inside a comment, so the value is always already 0 at address end.

Design doc

ARCHITECTURE.md documents the resulting parser internals — the dispatch loop, the 12 states → 7 handlers, the ParseContext anatomy and reentrancy rationale, and the per-address reset. The dispatch loop and state machine are mermaid diagrams and render inline on GitHub. It's the implementation companion to DESIGN.md (RFC semantics); linked from the README docs line.

Post-review updates

  • Renamed the parse-state local $emailAddress$ctx (481 sites) — it holds a ParseContext, not an address; the plural results array $emailAddresses is untouched.
  • Consolidated the per-address reset into resetAddress(int $state, int $subState) as the single source of truth. It now also clears state/subState/commentNestLevel — the last had no explicit reset before and only self-healed because a leading ( reassigns it to 1, a latent leak for any future per-address field.
  • validateLocalPart() kept fully backward compatible. Rather than change its signature (arrayParseContext), it retains its original array signature and the parser dispatches through it via a small legacy-array bridge — so it stays a live extension point and existing subclass overrides still fire (regression-tested in testDeprecatedValidateLocalPartOverrideStillTakesEffect). It's now @deprecated (removed in 4.0, where local-part validation folds into a private ParseContext-based method); ParseOptions is the supported way to customize validation. ParseContext is @internal.
  • Added a reentrancy regression test (testParserIsReentrantAcrossLocalPartNormalizer) — a normalizer that re-enters the same parser mid-parse; locks in the fresh-context-per-call invariant.
  • The snake_case field rename and three other cleanups surfaced in review are tracked under the delivered refactor item in ROADMAP.md, not blocking this merge.

Replace the ~24-key $emailAddress accumulator array threaded through
parse() and its validation helpers with a typed ParseContext object.
Property names mirror the former array keys so the change is a pure
mechanical conversion with no behaviour change.

A fresh ParseContext is created per parse() call and reset per address
via resetAddress(); it is never stored on the Parse instance, preserving
reentrancy across a localPartNormalizer callback.

Type the addAddress() parameters (array/ParseContext/int) and drop the
always-true isset() guard on the non-nullable domain property; refresh
the PHPStan and Psalm baselines to drop the now-obsolete array-shape
entries.
Decompose the ~772-line parse() state machine into a thin switch that
dispatches to one handler per parser state, plus sub-handlers for the
heavy STATE_ADDRESS branches (CFWS, '@', '.', atext, non-atext). parse()
is now ~190 lines.

Loop control (state/subState/commentNestLevel) and the hoisted input and
config move onto ParseContext so the handlers read them without long
parameter lists; behaviour, error codes and output are unchanged. Refresh
the Psalm baseline for the state-machine narrowing false-positives that
shift when the discriminant becomes a context property (PHPStan level 8
handles the mutation across calls and stays clean).
Fold the atext and period handling back inline into handleStateAddress so
the dominant STATE_ADDRESS path makes a single method call per character
instead of two. The larger, less-frequent branches (CFWS whitespace, '@',
non-atext) stay in their own helpers.

Behaviour is unchanged (full suite still green). Under opcache+JIT the
batch-parsing benchmarks now run at or below the pre-refactor baseline.
@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.75%. Comparing base (8a7b37c) to head (1ed22be).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master      #71      +/-   ##
============================================
- Coverage     96.01%   95.75%   -0.26%     
- Complexity      434      446      +12     
============================================
  Files             6        7       +1     
  Lines          1078     1108      +30     
============================================
+ Hits           1035     1061      +26     
- Misses           43       47       +4     
Files with missing lines Coverage Δ
src/Parse.php 94.06% <ø> (-0.53%) ⬇️
src/ParseContext.php 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Post-review cleanup for the parse() decomposition.

- Rename the parse-state local (and every handler parameter) from
  $emailAddress to $ctx: it holds a ParseContext, not an address, and
  the old name shadowed the concept of the address being built. Pure
  mechanical rename, no behavior change.

- Make ParseContext::resetAddress(int $state, int $subState) the single
  source of truth for per-address reset. It now also sets state/subState
  and zeroes commentNestLevel, which nothing reset before: an
  unterminated comment could leak its nesting level into the next address
  in a batch, self-healing only because '(' reassigns it to 1. Both the
  initial setup and the per-address reset in parse() now go through it.

Roadmap: mark the parse() readability refactor delivered and record the
remaining non-blocking follow-ups (snake_case fields, structural split of
the context's three concerns, chars/len duplication, handleStateAddress).

108 tests / 7199 assertions, PHPStan and CS clean.
Document the parser internals introduced by the parse() decomposition:
the character-by-character dispatch loop, the 12 states and their 7
handlers, the ParseContext object and how a fresh instance per call keeps
parse() reentrant, and the single-source-of-truth per-address reset.

State machine and dispatch loop are drawn as mermaid so they render inline
on GitHub. Complements DESIGN.md (RFC semantics) with the implementation
shape; linked from the README docs line.
…line local

Follow-up to the parse() decomposition review (PR #71).

- Mark validateLocalPart() @internal. It is protected on a non-final class,
  so its array -> ParseContext signature change is technically a subclass
  break; but it takes the parser's internal accumulator and was never a
  supported extension point (validation is customized via ParseOptions).
  Documented as such and slated to go private in v4.0 (roadmap).

- Give ParseContext a constructor requiring the initial state/subState,
  which runs resetAddress(). This makes an un-initialized context
  unrepresentable, replacing the "construct then remember to reset" pattern
  and the misleading zero-value field defaults (subState 0 = STATE_TRIM,
  not the required STATE_START).

- Inline the single-use $separators local straight onto the context.

108 tests / 7172 assertions, PHPStan and CS clean.
The file had drifted into a jumbled mix of shipped and planned work with
duplicated and misfiled sections. Reorganized without dropping substance:

- Split into three clear parts: Released (v3.1-v3.8 + deprecations + docs),
  Quality & infrastructure (continuous), and Planned (v4.0 + backlog).
- Removed the duplicate v4.0 list (the Deprecation Timeline had its own
  "v4.0 - planned" subsection overlapping the real v4.0 section); merged and
  deduped the API-cleanup items into a single v4.0 section.
- Refiled the shipped 3.8.0 homoglyph feature out of "v4.0 - Breaking" into
  Released; its target-list follow-up stays under Planned.
- Compressed the now-delivered parse() decomposition to a one-line record
  linking ARCHITECTURE.md, with the four review follow-ups moved to Backlog.
- Dropped the stale proposal text ("currently 99 tests", the pre-refactor
  complexity pitch) and the trailing stray footnote.
The decomposition changed where Psalm narrows $ctx->state, so 15 baseline
suppressions no longer match any code and CI (findUnusedBaselineEntry) fails
them: 1 InvalidCast, 1 ParadoxicalCondition, 1 RedundantCondition, and 12
TypeDoesNotContainType. Regenerated with --update-baseline (69 -> 54 entries);
no new suppressions added. Psalm, PHPStan L8, CS, and 108 tests all green.
Review-driven follow-ups on the parse() decomposition.

- Add a reentrancy regression test: a localPartNormalizer that re-enters the
  same parser mid-parse. Locks in the property the whole ParseContext design
  exists for — a fresh per-call context, never on the instance — so a future
  change that stored parse state on $this would fail here.
- Mark ParseContext @internal (its ~24-field shape is not a stable API) and
  fix an orphaned docblock: resetAddress()'s description had been stranded
  above __construct() when the constructor was added.
- CHANGELOG [Unreleased]: record the internal decomposition and the
  validateLocalPart() array -> ParseContext / @internal change.
- ROADMAP: expand the refactor follow-ups with the SOTA items surfaced in
  review — readonly snapshot/config fields, a ParserState backed enum
  (benchmark-gated), and decomposing addAddress().

109 tests / 7177 assertions, PHPStan L8, Psalm, CS all green.
Makes the parse() decomposition fully backward compatible: no public or
protected signature changes vs the previous release.

validateLocalPart() reads only local_part_parsed and local_part_quoted from
the accumulator, so instead of changing its signature to ParseContext it keeps
its original array signature and is dispatched through a small legacy-array
bridge at the one call site. It stays a live (virtual) extension point — a
subclass override is still invoked — but is now @deprecated and removed in
4.0, where local-part validation folds into a private ParseContext-based
method. The supported way to customize validation remains ParseOptions.

- Added a BC regression test: a Parse subclass overriding validateLocalPart()
  still changes the outcome (rejects a 'blocked' local part).
- ParseContext stays @internal.
- One @psalm-suppress DeprecatedMethod on the intentional internal BC hook.

110 tests / 7193 assertions, PHPStan L8, Psalm, CS all green.
The v4.0 removal was already listed under Planned; add the matching entry to
the Deprecations record so "what's deprecated / when removed" is complete.
@mmucklo
mmucklo merged commit 360a8ff into master Aug 26, 2026
13 of 14 checks passed
@mmucklo
mmucklo deleted the refactor/parse-decompose branch August 26, 2026 06:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant